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DETAILED ACTION 
Response to Amendment 

The amendment filed on 14 November 2005, has been entered and made of 
record. Claims 2 and 9 have been cancelled. Claims 1, 3-8, and 10-14 are pending. 

Response to Arguments 

Applicant's arguments with respect to claims 1-14 have been considered but are 
moot in view of the new ground(s) of rejection. 

Applicant argues Hanna does not teach the position coordinates of the image 
information are based on at least one of space coordinate of the camera, a zoom factor 
and a space vector. Examiner respectfully disagrees. Hanna estimates the pose of the 
detected logo pattern "A" by using the orientation, scale and perspective parameters of 
each of the sequence of successive 2D video image frames of the background scene 
with respect to the image of one or more reference objects in that video image frame 
[col. 4, lines 40-50]. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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1. Claims 1, 3-8, and 10-14 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Hanna et al. (US Patent No. 5,566,251) in view of Silverbrook (US 
Patent No. 5,479,205). 

Hanna et al. teaches the limitation of claims 1, 3-8, and 10-14 with the exception of 
disclosing a triggering unit and a display screen. However, Silverbrook discloses a 
video camera and recorder for combining a real-time image with a secondary image. 

In regards to claims 1, 7, 8 and 14 - 

Referring to Fig. 2, camera 200A records in real time a sequence of successive 2- 
dimensional video image frames of physical three-dimensional objects, such as 
billboard (202), situated in background scene (204A) (claims 1, 8: a camera configured 
to acquire image information of a section of an environment) [column 4, lines 18-22]. 
Billboard (202) is assumed immovable, but camera (200A) is moveable both to the left 
and right and toward and away from the objects as indicated by arrowed lines (206A). 
This will result in the orientation and scale of each of the sequence of video images of 
billboard (202) in background scene (204A) changing in correspondence with the 
movement of camera (200) from one position to another, as indicated by arrow lines 
(208A) (claims 1, 8: at least one of a zoom device configured to altera size of the 
section in accordance with a zoom factor and a device configured for three-dimensional 
orientation of the a camera in accordance with a space vector) [column 4, lines 24-33]. 
Means 21 OA performs the functions of (1) detecting logo pattern "A" and (2) estimating 
the pose of detected logo pattern "A" (i.e., estimating the orientation, scale and 
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perspective parameters of logo pattern "A" in each of the sequence of successive 2D 
video image frames of background scene (204A) with respect to the image of one or 
more reference objects, such as billboard (202) itself) in that video image frame (claims 
1,8: wherein the computer unit is configured to compute position coordinates of the 
image information based on at least one of space coordinates of the camera, the zoom 
factor, and the space vector, computer positions of representations of the image 
information on a display area of a display device; claims 7, 14: processing image 
information) [column 4, lines 43-49]. Additionally, means 214, responsive to a "pose 
information" (i.e., orientation, scale, perspective distortion, etc. parameters of detected 
logo patter "A") input (215) from means 210A and on-time selection via input (217) of 
manually-selected size, shape, orientation, etc. parameters of logo pattern "B" itself 
performs the function of geometrically transforming the orientation and scale of logo "B" 
of foreground scene (204B) to match the orientation and scale of the estimated pose of 
logo pattern "A" in the current image frame (as indicated by (216)) (claims 1,8: wherein 
the computer unit is configured to assign the user information to the position 
coordinates; claims 7, 14: processing use information) [column 4, lines 64-68; column 5, 
lines 1-6]. The operation of video switch (212) merges the video output of camera (200) 
with the geometrically-transformed logo pattern "B" to result in output scene (218), 
wherein logo pattern "A" is replaced by inserted logo pattern "B" (claims 1, 8: an image 
processing unit for processing the image information and the user information so as to 
reproduce the image information and the user information with the display device and 
so as to insert the user information in a location on the display area at the positions of 
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the representations of the image information having the position coordinates to which 
the respective user information is assigned; claims 7, 14: reproducing the image 
information and the user information with the display device, inserting the user 
information in the location on the display area in accordance with an imaging procedure) 
[column 5, lines 6-9]. 

Silverbrook discloses a camcorder system. Referring to Fig. 1 , the video 
camcorder system's circuitry (40) includes an alpha-numeric liquid crystal display (LCD) 
(61) and various miscellaneous controls (62) such as an auto zoom controller (66), and 
a manual zoom control (70) as seen in Fig. 2 (claims 1 , 8: comprises a triggering unit 
configured to trigger at least one of the camera, the zoom device, and the device for 3D 
orientation of the camera is accordance with at least one of the zoom factor and the 
space vector) [column 2, lines 23-30]. The circuitry (40) of the system receives user 
input via a keyboard (17) that includes an alphanumeric keypad (33) a touch panel (31), 
and animation control keys (38) as shown in Fig. 2. The keyboard (17) connects to a 
controller (16) that interprets commands and provides control commands onto a bus 
(25) that interconnects other components within the system (40) [column 2, lines 43-45]. 

Therefore, it would have been obvious to one of ordinary skill in the art at the time 
the invention was made to substitute the camera of Hanna et al. with the video 
camcorder system of Silverbrook where the output from the CCD of Silverbrook would 
be connected to means 21 OA of Hanna et al. so that the orientation and scaling of the 
camera within the combined invention is able to be user controlled. 
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In regards to claims 3 and 10 - 

The rationale of claims 1 and 8 respectively, are incorporated herein. Referring to 
Fig. 2 of Hanna et al., camera (200A) is moveable both to the left and right and toward 
and away from the objects as indicated by arrowed lines (206A). This will result in the 
orientation and scale of each of the sequence of video images of billboard (202) in 
background scene (204A) changing in correspondence with the movement of camera 
(200) from one position to another, as indicated by arrow lines (208A) [column 4, lines 
24-33]. Thus the movement of the camera toward the billboard (202) in the background 
scene (204A) and the scaling correspond to a zoom function. Means 21 OA performs 
the functions of (1) detecting logo pattern "A" and (2) estimating the pose of detected 
logo pattern "A" (i.e., estimating the orientation, scale and perspective parameters of 
logo pattern "A" in each of the sequence of successive 2D video image frames of 
background scene (204A) with respect to the image of one or more reference objects, 
such as billboard (202) itself) in that video image frame [column 4, lines 43-49]. 
Additionally, means 214, responsive to a "pose information" (i.e., orientation, scale, 
perspective distortion, etc. parameters of detected logo patter "A") input (215) from 
means 21 OA and on-time selection via input (217) of manually-selected size, shape, 
orientation, etc. parameters of logo pattern "B" itself performs the function of 
geometrically transforming the orientation and scale of logo "B" of foreground scene 
(204B) to match the orientation and scale of the estimated pose of logo pattern "A" in 
the current image frame (as indicated by (216)) (claims 3, 10: configured to select and 
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insert the user information as a function of the zoom factor) [column 4, lines 64-68; 
column 5, lines 1-6]. 

In regards to claim 4 and 11 - 

The rationale of claims 1 and 8 respectively, are incorporated herein. Referring to 
Fig. 2 of Hanna et al., the operation of video switch (212) mergers the video output of 
camera (200) with the geometrically transformed logo pattern "B" (i.e. user information 
as claimed) to result in output scene (218), wherein logo pattern "A" is replaced by 
inserted logo pattern "B". Thus, the pose of logo pattern "B" in output scene (218) 
changes in correspondence with the movement of camera (200) from one position to 
another, as indicated by arrowed lines (220) (claims 4, 11: user information comprises 
at least one of static and dynamic information) [column 5, lines 5-12]. 

In regards to claims 5 and 12 - 

The rationale of claims 1 and 8 respectively, are incorporated herein. Silverbrook 
discloses a camcorder system. Referring to Fig. 1 , the video camcorder system's 
circuitry (40) includes an alpha-numeric liquid crystal display (LCD) (61) (claims 5, 12: 
the camera comprises a video camera and the display device comprises a display 
screen) and various miscellaneous controls (62) such as an auto zoom controller (66), 
and a manual zoom control (70) as seen in Fig. 2 [column 2, lines 23-30]. 
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In regards to claims 6 and 13 - 

The rationale of claims 1 and 8 respectively, are incorporated herein. Silverbrook 
discloses a camcorder system. Referring to Fig. 1 , the video camcorder system's 
circuitry (40) includes an alpha-numeric liquid crystal display (LCD) (61) and various 
miscellaneous controls (62) such as an auto zoom controller (66), and a manual zoom 
control (70) as seen in Fig. 2 (said triggering unit) [column 2, lines 23-30]. The 
circuitry (40) of the system receives user input via a keyboard (17) that includes an 
alphanumeric keypad (33) a touch panel (31), and animation control keys (38) as shown 
in Fig. 2. The keyboard (17) connects to a controller (16) that interprets commands and 
provides control commands onto a bus (25) that interconnects other components within 
the system (40) (claims 6, 13: the triggering unit comprises an operator interface) 
[column 2, lines 43-45]. 

Conclusion 

THIS ACTION IS MADE FINAL, Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 . 1 36(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
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the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Michelle K. Lay whose telephone number is (571) 272- 
7661. The examiner can normally be reached on Monday - Friday, 7:00am - 4:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Michael Razavi can be reached on (571) 272-7664. The fax phone number 
for the organization where this application or proceeding is assigned is (571) 273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 




Michelle K. Lay 
Patent Examiner 
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